Appln.No.: 09/768,813 

Amendment dated February 23, 2006 

Reply to Office Action of November 23, 2005 

This listing of claims will replace all prior versions, and listings, of claims in the application:] 
Listing of Claims: 

Claim 1 (previously presented): A system for receiving audio input comprising: 
a display for displaying electronic information; 
an audio input receiving audio annotations; and, 

a processor for associating said received audio annotations with said displayed electronic 
information, wherein said processor further associates an author with each of said audio 
annotations, each of said audio annotations being randomly accessible based on the author. 

Claim 2 (currently amended): The system according to claim 1, wherein each of said audio 
annotations is in the form of audio clips. 

Claim 3 (canceled) 

Claim 4 (previously presented): The system according to claim 1, further comprising: 
a storage for storing each of said audio annotations with the associated author. 

Claim 5 (previously presented): The system according to claim 1, further comprising: 
an input receiving a user's input, 

wherein said processor starts recording an audio annotation from said audio input in 
response to said user's input. 

Claim 6 (previously presented): The system according to claim 1, wherein said processor 
includes a voice activated recording system for recording said audio annotation. 

Claim 7 (previously presented): The system according to claim 6, wherein said voice activated 
recording system records when said audio annotation exceeds a predetermined threshold. 



Page 2 of 18 



Appln.No.: 09/768,813 

Amendment dated February 23, 2006 

Reply to Office Action of November 23, 2005 

Claim 8 (previously presented): The system according to claim 6, wherein said voice activated 
recording system records when a known user's voice is detected in said audio annotation. 

Claim 9 (previously presented): The system according to claim 1, wherein said processor 
controls said display to indicate that an audio annotation is associated with said displayed 
electronic information. 

Claim 1 0 (previously presented): A system for playing audio content, said system comprising: 
a display for displaying electronic information; 

a storage for storing audio annotations, each of said audio annotations including an author 
and being associated with said displayed electronic information, each of said audio annotations 
being randomly addressable based on the author; 

an output for outputting at least some of said audio annotations responsive to navigation 
of said displayed electronic information; and 

a processor for controlling said display, said storage and said output. 

Claim 11 (previously presented): The system according to claim 10, wherein said audio 
annotations include audio clips. 

Claim 12 (canceled) 

Claim 13 (currently amended): The system according to claim 4310, wherein said storage is a 
database. 

Claim 14 (previously presented): The system according to claim 10, further comprising: 
an input for receiving a user's input, 

wherein said output outputs at least some of said audio annotations in response to 
receiving said user's input. 
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Claim 15 (previously presented): The system according to claim 10, further comprising: 
an input for receiving a user's input of an author, 

wherein said processor searches authors of said stored audio annotations in response to 
said user's input. 

Claim 16 (original): The system according to claim 15, wherein the output of said processor is 
sent to said display to display an indication of the search results. 

Claim 17 (previously presented): The system according to claim 15, wherein the output of said 
controller is sent to the output for playing an audio annotation with an author matching the 
search results. 

Claim 18 (previously presented): The system according to claim 10, wherein said processor 
retrieves all audio annotations associated with said electronic information when said electronic 
information is accessed. 

Claim 19 (previously presented): The system according to claim 10, wherein said processor 
outputs selected audio annotations to be played through said output when a page of said 
electronic information is displayed. 

Claim 20 (previously presented): The system according to claim 19, wherein said processor 
automatically plays said selected audio annotations when said page is displayed. 

Claim 21 (previously presented): The system according to claim 10, further comprising: 
a communication link to transmit said audio annotations. 

Claim 22 (previously presented): The system according to claim 21, further comprising: 

a network connected to said communication link for receiving said audio annotations, 
said network being accessible by other users. 
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Claim 23 (previously presented): The system according to claim 21, further comprising: 

a receiving device of another user for receiving said audio annotations, said receiving 
device receiving said audio annotations through one of a wired or wireless interface. 

Claim 24 (previously presented): The system according to claim 22, wherein said network 
further processes said audio annotations. 

Claim 25 (previously presented): The system according to claim 22, wherein said network 
includes a database for storing said audio annotations. 

Claim 26 (previously presented): The system according to claim 22, wherein said network 
receives audio annotations without receiving said electronic information associated with said 
audio annotations. 

Claim 27 (currently amended): A user interface for displaying electronic information to a user 
comprising: 

a first display portion for displaying a portion of a document; and 

a second display portion for displaying a graphical indication that said document includes 
an audio annotation associated with said displayed portion of said document^ 
wherein said audio annotation is associated with an author . 

Claim 28 (original): The user interface according to claim 27, further comprising: 
a third display portion for displaying a non-audio annotation. 

Claim 29 (original): The user interface according to claim 27, further comprising: 

a third display portion for displaying an indication that said audio annotation is being 
recorded or played back. 

Claim 30 (original): The user interface according to claim 27, further comprising: 

a third display portion for displaying one of a document tape or a master tape. 
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Claim 31 (original): The user interface according to claim 27, further comprising: 

a third display portion for receiving a user input of a property or properties of said audio 
annotation. 

Claim 32 (original): The user interface according to claim 27, wherein said audio annotation is 
recordable by said user. 

Claim 33 (currently amended): A process for recording an audio annotation comprising the steps 
of: 

displaying electronic information; 
receiving a user input; 

recording an audio annotation in response to said user input; and 
associating an author of the audio annotation with the audio annotation; and 
associating said audio annotation with properties including a displayed portion of said 
electronic information. 

Claim 34 (original): The process according to claim 33, further comprising the step of: 

storing said audio annotation prior to the association of said audio annotation with said 
displayed portion. 

Claim 35 (original): The process according to claim 33, further comprising the step of: 

storing said audio annotation after the association of said audio annotation with said 
displayed portion. 

Claim 36 (original): The process according to claim 33, wherein said recording step records all 
ambient sounds. 

Claim 37 (original): The process according to claim 33, wherein said recording step records only 
sounds above a predetermined threshold. 
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Claim 38 (original): The process according to claim 37, wherein said recording step records only 
a specific user's voice. 

Claim 39 (original): The process according to claim 33, further comprising the step of: 

associating additional properties with said audio annotation at the start of recording of 
said audio annotation. 

Claim 40 (original): The process according to claim 33, wherein one of said properties is a file 
position or document position of an item on said displayed portion of said electronic information. 

Claim 41 (original): The process according to claim 33, wherein one of said properties is a start 
identification of said displayed portion of said electronic information. 

Claim 42 (original): The process according to claim 33, further comprising the steps of: 
storing said audio annotation; and, 

searching audio annotations including said audio annotation for at least one property 
matching a query. 

Claim 43 (currently amended): A process for playing audio annotations comprising the steps of: 
displaying a portion of electronic information; 
receiving a user input; 

retrieving audio annotations , each of said audio annotations being associated with an 

author ; 

assembling said audio annotations into an audio stream; and 
playing said audio stream. 

Claim 44 (original): The process according to claim 43, further comprising the step of: 
waiting for a second user input prior to playing said audio stream. 
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Claim 45 (previously presented): The process according to claim 43, further comprising the step 
of: 

playing said audio stream once said audio stream is assembled. 

Claim 46 (original): The process according to claim 43, wherein said user input is a text query. 

Claim 47 (original): The process according to claim 43, wherein said user input is a voice query. 

Claim 48 (original): The process according to claim 43, further comprising the steps of: 

altering the display of said portion to match a currently playing annotation in said audio 

stream. 

Claim 49 (previously presented): The process according to claim 48, wherein said altering step 
includes the steps of: 

comparing the length of said currently playing annotation with starting identifications of 
displayable portions of said electronic information; and 

displaying the portion of said electronic information supporting the greater length of said 
currently playing annotation. 

Claim 50 (currently amended): A process for playing audio annotations comprising the steps of: 
navigating to a page; 

retrieving at least one audio annotation associated with a page or associated with an item 
on a page; and 

playing said at least one audio annotation 

wherein said at least one audio annotation is associated with a property identifying an 
author of said at least one audio annotation . 

Claim 51 (original): The process according to claim 50, further comprising the step of: 
waiting for a user input prior to playing said audio annotation. 
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Claim 52 (original): The process according to claim 50, wherein said item on said page includes 
at least one of embedded notes, inked notes, highlights and underlining. 

Claim 53 (original): The process according to claim 50, wherein said at least one audio 
annotation was previously retrieved and said retrieving step includes indexing said previously 
retrieved at least one audio annotation. 

Claim 54 (original): The process according to claim 50, wherein said at least one audio 
annotation is the result of a newly executed query. 

Claim 55 (currently amended): A computer readable medium having a data structure stored 
thereon, said data structure comprising: 

a document; 

a link object; and 

audio content with at least one property, 

wherein said link object references said document and references said audio content^ 

wherein said audio content includes at least one audio annotation, 

wherein at least one property identifies an author of the at least one audio annotation . 

Claim 56 (currently amended): The data structure according to claim 55, wherein said- at least 
one p roperty relates to the time said audio content started recording. 

Claim 57 (currently amended): The data structure according to claim 55, wherein said- at least 
one p roperty relates to the time said audio content stopped recording. 

Claim 58 (currently amended): The data structure according to claim 55, wherein said -at least 
one p roperty relates to the length of recording of said audio content. 

Claim 59 (canceled) 
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Claim 60 (currently amended): The data structure according to claim 55, wherein said -at least 
one p roperty relates to the-a_start ID. 

Claim 61 (currently amended): The data structure according to claim 55, wherein said -sat least 
one property relates to the-a_stop ID. 

Claim 62 (original): The data structure according to claim 55, wherein said audio content is 
comprised of a plurality of audio clips. 

Claim 63 (original): The data structure according to claim 62, wherein said audio clips are 
stored in a database. 

Claim 64 (currently amended): The data structure according to claim 55, wherein said- at least 
one p roperty is one of a_plurality of properties and said properties are in a marked up language 
form. 

Claim 65 (original): The data structure according to claim 64, wherein said properties are in 
XML. 

Claim 66 (original): The data structure according to claim 55, wherein said audio content is 
stored within a document. 

Claim 67 (original): The data structure according to claim 55, wherein said audio content is 
stored apart from a document. 

Claim 68 (currently amended): The data structure according to claim 67, wherein said audio 
content is stored in a database with at least one property designating the-aj>osition of a_viewed 
document relating to said audio content. 
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Claim 69 (currently amended): The data structure according to claim 67, wherein said audio 
content is stored in a database and linked to a separate annotation document that stores the-a 
position of a viewed document relating to said audio content. 

Claim 70 (currently amended): A process for recording audio content comprising the steps of: 
navigating to a page of a document; 
recording said audio content; and 

associating properties with said audio content such that retrieval of said audio content 
positions said audio content after previously recorded audio content, 

wherein said audio content includes at least one audio annotation, 

wherein at least one property identifies an author of the at least one audio annotation . 

Claim 71 (previously presented): The process according to claim 70, wherein said audio content 
comprises audio clips and wherein said properties include a time property. 

Claim 72 (original): The process according to claim 71, wherein said audio content and said 
previously recorded audio content is ordered at least by said time property. 

Claim 73 (currently amended): A process of searching audio clips comprising the steps of: 
inputting search terms or properties; 

searching said audio clips for said search terms or properties; and 
ordering audio clips detected by said searching step for output^ 

wherein each of said audio clips includes an audio annotation associated with an author 
of the audio annotation, 

wherein said search terms or properties identify an author of the audio annotation . 

Claim 74 (original): The process according to claim 73, wherein said inputting step further 
comprises the steps of: 

receiving verbally delimited keywords; and 

converting said verbally delimited keywords into search terms or properties. 
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Claim 75 (currently amended): A process for recording audio information comprising the steps 
of: 

recording audio signals as a first file; 
processing said file to extract audio clips; and 
storing said audio clips, 

wherein said processing separates the content of said first file into audio clips based on 

events^ 

wherein each of said audio clips includes an audio annotation associated with an author 
of the audio annotation, 

wherein said processing associates an author of the recorded audio signal with the audio 
annotation . 

Claim 76 (original): The process for recording according to claim 75, 
wherein said audio signals include speech, and 

wherein said events comprise at least one of short pauses in said speech, a pause of a 
predetermined length, and a user navigating away from a displayed page. 

Claim 77 (currently amended): A process for associating audio notes and handwritten notes 
comprising the steps of: 

creating a handwritten note; 

associating a time at which said handwritten note was created with said handwritten note; 
creating an audio note; and 

associating a time at which said audio note was created with said audio note ; and 
associating an author of said audio note with said audio note , 

wherein, upon selection of said handwritten note, audio notes created at or near the time 
at which said handwritten note was created are located with said handwritten note. 

Claim 78 (original): The process according to claim 77, wherein locating said audio notes 
includes the step of querying a database for audio clips. 



Page 12 of 18 



Appln.No.: 09/768,813 

Amendment dated February 23, 2006 

Reply to Office Action of November 23, 2005 

Claim 79 (original): The process according to claim 77, wherein locating said audio notes 
includes the step of searching a table. 

Claim 80 (original): The process according to claim 77, wherein locating said audio notes 
includes the step of searching a linked list. 

Claim 81 (original): The process of claim 77, wherein said audio notes are comprised of audio 
clips in which each audio clip has a time of creation associated with each audio clip. 

Claim 82 (original): The process according to claim 77, further comprising the step of: 
playing said audio notes. 

Claim 83 (currently amended): A process for playing audio notes comprising the steps of: 
displaying a first page of electronic information; 
playing audio notes associated with said first page; 
displaying a second page of electronic information; and, 
playing audio notes associated with said second page A 

wherein each audio note is associated with a property identifying an author of said audio 

note . 

Claim 84 (original): The process according to claim 83, further comprising the step of receiving 
user input, 

wherein, in response to said user input, said second page is displayed. 

Claim 85 (currently amended): A process of recording audio notes comprising the steps of: 
displaying a first page of electronic information; 
recording a first set of audio notes; 

associating said first set of audio notes with said first page; 
displaying a second page of electronic information; 
recording a second set of audio notes; and 
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associating said second set of audio notes with said second page^ 

wherein each set of audio notes is associated with a property identifying an author of said 
set of audio notes . 

Claim 86 (original): The process according to claim 85, further comprising the step of receiving 
user input, 

wherein, in response to said user input, said second page is displayed. 
Claims 87-88 (canceled) 



Page 14 of 18 



